Skip to content

Revert "DEP: de-duplicate and sort optional dependencies (#2021)"#2026

Merged
martindurant merged 2 commits into
fsspec:masterfrom
martindurant:rev
Apr 29, 2026
Merged

Revert "DEP: de-duplicate and sort optional dependencies (#2021)"#2026
martindurant merged 2 commits into
fsspec:masterfrom
martindurant:rev

Conversation

@martindurant
Copy link
Copy Markdown
Member

This reverts commit 3bc67f8.

@martindurant martindurant merged commit f58bc85 into fsspec:master Apr 29, 2026
11 checks passed
@martindurant martindurant deleted the rev branch April 29, 2026 20:17
@neutrinoceros
Copy link
Copy Markdown
Contributor

@martindurant why was this reverted ? I'd love to help with fixing any problems caused by my patch

@martindurant
Copy link
Copy Markdown
Member Author

I caused errors like this after merge. I don't know why, but I had to get a release out.
https://github.com/fsspec/filesystem_spec/actions/runs/25130463942/job/73655009945

ERROR: Cannot install fsspec, fsspec 2026.3.0.post10+g3bc67f85c (from /home/runner/work/filesystem_spec/filesystem_spec) and fsspec[test-full]==2026.3.0.post10+g3bc67f85c because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested fsspec 2026.3.0.post10+g3bc67f85c (from /home/runner/work/filesystem_spec/filesystem_spec)
    fsspec[test-full] 2026.3.0.post10+g3bc67f85c depends on fsspec 2026.3.0.post10+g3bc67f85c (from /home/runner/work/filesystem_spec/filesystem_spec)
    gcsfs 2026.3.0 depends on fsspec==2026.3.0
    gcsfs 2026.2.0 depends on fsspec==2026.2.0
    gcsfs 2026.1.0 depends on fsspec==2026.1.0
    gcsfs 2025.12.0 depends on fsspec==2025.12.0
    gcsfs 2025.10.0 depends on fsspec==2025.10.0
    gcsfs 2025.9.0 depends on fsspec==2025.9.0
    gcsfs 2025.7.0 depends on fsspec==2025.7.0
    gcsfs 2025.5.1 depends on fsspec==2025.5.1
    gcsfs 2025.5.0.post1 depends on fsspec==2025.5.0
    gcsfs 2025.5.0 depends on fsspec==2025.5.0
    gcsfs 2025.3.2 depends on fsspec==2025.3.2
    gcsfs 2025.3.1 depends on fsspec==2025.3.1
    gcsfs 2025.3.0 depends on fsspec==2025.3.0
    gcsfs 2025.2.0 depends on fsspec==2025.2.0
    gcsfs 2024.12.0 depends on fsspec==2024.12.0
    gcsfs 2024.10.0 depends on fsspec==2024.10.0
    gcsfs 2024.9.0.post1 depends on fsspec==2024.9.0
    gcsfs 2024.6.1 depends on fsspec==2024.6.1
    gcsfs 2024.6.0 depends on fsspec==2024.6.0
    gcsfs 2024.5.0 depends on fsspec==2024.5.0
    gcsfs 2024.3.1 depends on fsspec==2024.3.1

Additionally, some packages in these conflicts have no matching distributions available for your environment:
    fsspec

@neutrinoceros
Copy link
Copy Markdown
Contributor

I see. There's a cycle in the dependency graph that's probably tricky for resolvers to work out because gcsfs expects an exact version of fsspec, which itself has a dynamic version number which will never match. I'm happy to investigate further, because I'm curious to know why resolution only fails with my patch, and could be hinting at a bug in the resolver itself. Double checking though: is is pip ?

@neutrinoceros
Copy link
Copy Markdown
Contributor

(Just checked the logs. It's pip 26.0.1)

@neutrinoceros
Copy link
Copy Markdown
Contributor

I can reproduce the error locally with pip 26.1, though what's unsettling is that I get the same result with or without my patch. I'll to better understand how it's used in CI.

@neutrinoceros
Copy link
Copy Markdown
Contributor

neutrinoceros commented May 3, 2026

to be more accurate:

  • python -m pip install ".[full_test]" works locally with pip 26.0.1 with and without my patch
  • python -m pip install ".[full]" fails locally with pip 26.0.1 regardless of wether my patch is present

Results do not change with pip 26.1
It's really not clear why results in CI differ, or why my patch would influence this. I'm still suspecting a bug in pip. I'll return to this.

@neutrinoceros
Copy link
Copy Markdown
Contributor

Maybe I should also have specified what platform I was running on. My previous tests were on macOS arm64. I'm now testing with Linux + x64, and already I'm getting different results:

on master

  • pip 26.1 + test_full: error
  • pip 26.1 + full: error
  • uv 0.11.8 + test_full: ok
  • uv 0.11.8 + full: error

@neutrinoceros
Copy link
Copy Markdown
Contributor

The plot thickens: it look like in all cases where my install succeeds, I end up with gcsfs v0.8.0, which is an ancient version and I'm guessing, the last one that didn't require an exactly pinned version of fsspec, that is, before v2026.4.0, which uses a lower bound instead of an exact pin (this is much easier for resolvers to work with). For reasons that are still unclear to me still, CI here manages to get v2026.4.0, but none of my local tests do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants